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dealing with MRI brain tumour. The results clearly indicate better 
performance in segmenting brain tumours than existing ones. 
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1. INTRODUCTION 

While performing neurological classification of Parkinson's and Alzheimer's disease, clinical 
physicians primarily rely upon magnetic resonance images (MRI) for effective segmentation of brain tissues. 
But, while classifying MRIs of the brain tissues, the clinical researchers have to invariably struggle with certain 
inhibiting factors like low contrast, non-uniformity, and complex structure, right from the stage in which an 
image is acquired. However, in a much-needed relief, in a majority of cases now machine learning is being 
applied as a technique while performing MRI segmentation [1]. In many landmark cases, deep learning (DL) 
methods have successfully demonstrated application of these convolutional neural network (CNN) based 
techniques in the realm of brain tissue segmentation. In essence, three layers viz., the fully connected layer, 
pooling as well as convolution layer form primary building blocks of a regular CNN framework. 

Moreover, the experience of earlier researchers has shown that it is a daunting and exhaustive task to 
recognize image paths in a typical CNN. The fully convolutional network (FCN), an end-to-end segmentation 
method has been introduced which makes a pixel-by-pixel prediction to produce the labels directly through 
analysis of images. In fact, on account of its better capacity to represent features, it produces better results in 
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object detection, classification, and segmentation, though with availability of sufficient training data. In recent 
times, the FCN network has been extended through more recycling extra feature maps to generate promising 
results for U-net. It has done so with even smaller amounts of training samples. However, its computational 
inefficiency is a cause of concern. The brain images are effectively segmented using a computational DL 
technique capable of performing even on scanty training data. SegNet and U-Net are combined as hybrid 
architecture. A highly performing multi-scale information is generated using the SegNet architecture as the 
base, while skip connection is used at the select de-convolution layer. The model yields faster convergence 
when pooling indices pass into de-convolution layers. Then, counters of brain tissue boundaries are extracted 
by combining additional level set layer within this architecture. Final validation is done through real brain 
tumour data set. Section 2 reviews the relevant literature, while suggested method is presented and discussed 
in detail in section 3. The final part 4 deals with presentation of the experimental results, which is discussed in 
detail as well as the conclusion. 


2. RELATED WORK 

Machine learning methods like the clustering and segmentation methods yield better results in tumour 
segmentation [1]-[3] used random forests (RF) to retrieve statistical features that are passed as input. In 2015, 
the best performer in the 2015 brain tumor segmentation (BraTS) challenge pioneered the application of 
convolutional neural networks to segment brain tumour images [4]. Some local features and global features are 
explored at the same time in this CNN architecture, clocking 30 times more speed than leading-edge solutions 
of the time. In addition, the convolutional implementation of a fully connected layer is applied in this 
architecture thus attaining 40 times acceleration. A CNN 3D architecture extracting patches of 3D voxels with 
varying brain MRI modalities [5]. 3D voxels are fed into a 4-layered CNN architecture to predict the tissue 
label of the centre voxel. More computational demands are prevented in 3D voxels by transforming 4D data 
into 2D data [6]. So that segmentation of brain tumours could be performed by 2D-CNN architecture. 11 CNN 
Architecture layers are evaluated on the BraTS dataset in [7] when small filters (3x3) are fit into convolutional 
layers and comparative dice scores are reported. In alignment with classification or clustering methods, CNNs 
limit the problems in training data and improve performance [8]. An effective deep learning-based approach, 
known as DMRes, an improvised version of deep medic was developed for segmenting brain tumour [9], [10]. 
The early contributors attained some degree of efficacy by combining Level set evolution and global 
smoothness with flexible topology changes and mathematical morphology [11]. It gave them certain scoring 
points over traditional statistical classification. The method evaluated the working of the algorithm based on 
volume overlap and Hausdorff distance. While setting the equation parameters in speed function, level set 
algorithms face some difficulty. 3D tumour segmentation (TLS) was done by applying level sets through a 
threshold-based scheme [12]. The speed function is designed using a global threshold on the basis of confidence 
interval with iterative updates (search-based and adaptive) in which users are involved in varied degrees in the 
evolution process. A signed pressure function (SPF) with efficiency to block contours at weak or blurred edges 
was done [13]. The algorithm differentiates tumours from the rest of an image by using local statistics. 
Additionally, automatic calculation of image thresholding is done here. 

Deep learning-based methods have been widely applied to many fields and have achieved state-of- 
the-art performance. However, brain tumour segmentation poses several unique challenges. First, image quality 
has a critical impact on segmentation performance [14]. For example, blurred images result in poor outcomes. 
Second, image pre-processing steps also have an impact on the performance. For example, intensity 
normalization across cases is critical for tumour segmentation. Third, tumour tissue heterogeneity may pose a 
serious challenge to the developing an effective method. Finally, data imbalance is common and poses another 
intricate challenge for the use of deep learning [15]. 


3. PROPOSED METHOD 

The proposed hybrid U-SegNet implementation as shown in the Figure 1. The input image is first 
entered into at the pre-processing stage and then trained by using hybrid CNN. Then, the level set extension 
(ELS) is integrated into Hybrid U-SegNet for extraction of boundaries from brain MRI images [16], [17]. 


3.1. U-SegNet architecture 

U-SegNet architecture, a hybrid structure formed by using SegNet and U-Net architectures is 
projected in Figure 2. As local information scores over global information in separating white matter (WM), 
gray matter (GM), and cerebrospinal fluid (CSF), axial slices of size 256*128 are assigned patch-based training. 
After observing the GM structures and noting the same in the chosen resolution dataset, local structures are 
formed on a patch size of 40 for segmentation. Input patches of 40*40*3 are easily handled by the SegNet 
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architecture at its best. A 3x3 kernel is used in each convolutional layer within the architecture in alignment 
with max-pool layers of size 2*2, besides activating rectified linear unit (ReLU) activation functions. A skip 
connection of U-Net type is inserted just at the upper layer for clearly displaying characteristic maps as shown 
in Figure 2. Here, coarser and finer information is consolidated by using a 1x1 convolutional layer for 
segmenting purpose, besides transferring lesser parameters to the last convolutional layer. Similar to U-Net, 
fine information is incorporated without increasing the parameters through the skip connection. In the end, the 
4-label classification as background (0), GM (1), WM (2) and CSF (3) is implemented by applying a SoftMax 
layer with 4 outputs as shown in Figure 2. 


Figure 1. Block diagram of the purposed method 
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Figure 2. Fully convolutional network (FCN) 


3.2. Level set 

Image segmentation [18] makes wide application of level set (LS) method with active contouring 
(AC) as it performs automatic manoeuvring of possible topological changes. The potential of LS in attaining 
accuracy in segmenting brain tumour is documented in [12], [13]. 


3.2.1. Background 
Consider the problem of segmentation of binary images in the dimension of 2D and to be denoted as 


Q. Next, also consider C will be the boundary of desired open set and it to be defined as C=OW, where W EQ. 
Now, [19] in case of the concept of LS framework the open set border to be defined as C to be defined with @ 
the level zero game and is represented as shown in (1): 


C= (X,Y): 6% Y) = 0} 


V(X,Y) = 24 in(C) = {(X,Y): (X,Y) > 0} (1) 
out(C) = {(X,Y):$(X%,Y) < 0} 
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The segmentation task in which Q to be represented as complete domain corresponds to I original image. Next, 
LS function to be denoted as ¢ and it grouped region Q into two regions: inside region of W to be in(C) and 
outside W to be defined as out(C). The length of the outline C is defined as shown in (2): 


Length(C) = | VHE, Y))I dxdy = | SODIO, NI dxdy (2) 
a a 
and the area inside the contour C is defined in 
Area(C) = | new, Y)) dxdy (3) 
a 


Typically, in a LS-based segmentation method, the beginning us made with level set o for the input image J. 
The through gradient descent is applied to initialize the update process in LS wherein an energy function that 
is a function of the variation in image attributes like colour and texture between the foreground and the 
background is minimized. The LS requires form and regions to improve performance. Since LS uses only low- 
level features, it is limited to reading complex images [20], [21]. However, deep networks are adept at learning 
and encoding critical features, thus helping in overcoming this limitation. 


3.2.2. Convolution layer 

In case of the convolution layer both the input and output to be represented as feature map. However, 
it is noticed that convolution output feature map is evaluated by performing the operation convolution with the 
feature map corresponds to the input layer and it is defined in (4). 


Ysa) = fs(X,0) =X * Ws + b (4) 


From the (4) X is to be considered as the convolution input layer feature map and b noticed as the bias of the 
convolution kernel. Similarly, W; represented as the convolution at a stride and it to be represented as s. Finally, 
the output of the convolution layer feature map to be defined as Y5.9and is formed with the convolutional 
layers of stride S and parameter 0 and the result is down sampled. 


3.2.3. Deconvolution layer 

The input feature maps are oversampled by deconvolutional layer when maximum pooling indices 
from the related convolutional feature map is applied. The output of the deconvolutional feature map is 
concatenated with the matching convolutional feature map through a skip connection. The feature maps in a 
deconvolutional layer are used as output ¥(5 9) picked from the preceding convolutional layer as its matching 
feature map. Let G,;(; T) denotes a deconvolutional layer parameterized by t which the convolution layer input 
with the factor and to be represented as s. Now, the output is resulted by concatenating relevant Convolutional 
Layer and is denoted as Ys_1,9) using the skip connection and is defined as (5). 


A 
Ys,r) = concat|Gs(Ys,@); T), Ys-1,6| (5) 


3.2.4. Hybrid U-SegNet with level set layer 

Inspired from the existing frameworks mentioned as U-SegNet and the LS framework, proposed 
hybrid method combined with above two methods and the output feature maps are represented as (Y) and is 
derived from U-SegNet using the euclidean distance transformation (€). The desired level set function to be 
represented in the (6). 


p=; 0Y)-$(1-Y) (6) 


From (6) the input space is represented as @. In order to minimize the energy function in the network model as 
shown in (7) is used and is defined as: 


ECC, $) = funo + u5(~)|Vb| + a(H ($) — GT)? 


° + Ay|H() - GPH) + AglH($) - GPA 
- H($))dxdy (7) 
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From the (7), it is observed that initial part denotes the inner contour and second part defied as the 
length of the contour C. Moreover, it is noticed that first part to be neglected in case of the u=0. Also, with 
similar to standard VLS,v > 0 and it is useful for the noise free. But the study introduced v <0 on case of the 
brain tumour. Next, the third part is represented as ground truth images and complete part is neglected in case 
of the a > 0. At final, the last two parts related the energy in and out contour C. In case of the brain tumour, in 
and out counters are defined as 2, and A, and also these values to be always positive. Finally, the two constants 
Cı and C; are specified in (8) and (9) and are used to optimize the energy function ¢ as shown in Figure 3. 


= Sg H@ay)H(p)axdy 
Jot (b)dxdy 


C (8) 


Z Jo HP) G.y)A-H(p))dxdy 
JqQ-H(o))dxdy 


C (9) 
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Figure 3. Proposed hybrid method 


4. EXPERIMENT RESULT 
4.1. Dataset and measurement 

BraTS data forms the training data for the new diagnostic reference levels (DRLs) method. It also 
applies BraTS 2013 and BraTS 2015 datasets to prove its better show than standard techniques. Also, BraTS 
2017 dataset is provided by medical image computing and computer-assisted intervention (MICCAI) for 
automated brain tumour segmentation task. Each dataset contains two subsets corresponding to low-grade 
glioma (LGG) and high grade gliomas (HGG) [22]. The dataset comprises training and testing datasets at 80% 
and 20%, respectively. BraTS 2017 uses 168 HGG and 60 LGG training set to test this network that is later 
applied BraTS 2015 and 2013 datasets. 


4.2. Evaluation metrics 
The validation of this method is validated using metrics as detailed below: 


4.2.1. True positive 

‘Sensitivity’ or ‘recall’ represent true positives, denoting that detects the condition when such a 
condition is there [23]. The True positive rate measure is proportion of the relation of both true positive rate as 
well as the addition of false negative rate and true positive rate. 
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4.2.2. True negative 

True negative is also termed as ‘specificity’. When as fraction of true negative are added to true 
negative and false positive, we notice true negative. It denotes the test result that does not identify the condition 
when the condition is present. 


TNR = TN / (FP +TN) 


4.2.3. False negative 
The proportion of false negative and the sum of true negative and false negative is referred to as false 
negative rate. It represents test result that detects the condition when the condition is absent. 


Ppv = a/ (a+c) 


4.2.4. False positive 

The fraction of false positive and the sum of false positive and true positive is added to get false 
positive rate. It indicates the test result that does not identify the condition when the condition is not present or 
absent. 


FPR = FP / (TP + FP) 


4.2.5. Positive predictive value 
In case of a likelihood of a positive test when disease is present in the body of the patient, then positive 
predictive value is derived 


Ppv = a/ (a+c) 


4.2.6. Negative predictive value 
Negative predictive is defined as the possibility of no presence of the disease, which can be a harmful 
test. 


NPV =d/b+d 


4.2.7. Dice similarity coefficient (DSC) 

Dice coefficient is one of the methods to establish the extent of the latitudinal connection between 
two binary images. The segmentation process is commonly used as the performance measures of dice 
coefficient, which provides more weighting to the instances [24], [25]. These values also range between zero 
and one. Dice coefficient is utilized to find the match between two similarities labelled as region (A and B) in 
the images, which is computed as: 


D = ey, x 100 
~ AG +AG 
2TP 


DSC = Ep E OTP + FN 


The quantitative evaluation based on Sensitivity (SEN) and Specificity (SPE) is given by the equation: 


SEN = one 
~ TP +FN 
SPE = = 
~ TN + FP 


4.3. Results and discussion 

The proposed algorithm achieved average dice scores of 0.89, 0.79, and 0.74 for whole tumour (WT), 
core tumour (CT) and enhancing CT regions, respectively in the HGG case on the 2013 BraTS dataset. In the 
case of LGG, the proposed model achieved dice scores of 0.89, 0.62, and 0.43 for WT, CT, and enhancing CT 
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regions. Thus, it has achieved high performance, which is better than the other standard methods. The 
sensitivity and specificity values achieved on BraTS 2013 dataset are 0.90, 0.89, and 0.93 and 0.92, 0.84, and 
0.86, respectively. Similarly, in case of LGG, the sensitivity and specificity values achieved on BraTS 2013 
dataset are 0.89, 0.83, and 0.85 and 0.91, 0.80, and 0.84, respectively. 

Based on the 2015 BraTS dataset, WT, CT, and enhancing CT regions receive dice score, 0.82, 0.73 
and 0.68 in HGG case. In case of LGG, for WT, CT, and enhancing CT regions receive dice scores of 0.82, 
0.57, and 0.40 in this model. Thus, it has achieved high performance, which is better than the other standard 
methods. The sensitivity and specificity values achieved on BraTS 2013 dataset are 0.83, 0.82, and 0.86); and 
0.85, 0.77, and 0.79, respectively. Similarly, in case of LGG, the sensitivity and specificity values achieved on 
BraTS 2013 dataset are 0.82, 0.76, and 0.78; and 0.84, 0.74, and 0.77, respectively. 

As shown in Table 1 and Table 2, there are consistent better dice scores in sensitivity and specificity 
produced by the proposed algorithm. The performance can be credited to the availability of additional training 
data from both 2013 and 2015 BraTS dataset that helped in fine tuning the hyper-parameters of this algorithm. 
Such as reliability against outliers, speed makes this algorithm achieve better segmentation of core tumour. 
Figure 4 shows the result of proposed method, quantitatively and shows that the proposed method produces 
best segmentation results as compared to the standard U-Net method. 


Table 1. Performance of proposed method vs standard methods tested via BraTS 2013 dataset 
Dice Score Sensitivity Specificity 
WT CT ET WT CT ET WT CT ET 
HGG 0.88 0.76 0.73 0.87 0.79 0.80 0.89 0.79 0.68 
LGG 0.65 0.53 0.00 0.71 0.75 0.72 0.75 0.73 0.69 
HGG 0.88 0.79 0.73 0.62 0.68 0.72 0.82 0.78 0.80 
LGG 0.88 0.58 0.21 0.78 O81 0.82 0.76 0.79 0.81 
HGG 0.89 0.79 0.74 0.90 0.89 0.93 0.92 0.84 0.86 
LGG 0.89 0.62 0.43 0.89 0.83 0.85 0.91 0.80 0.84 


Pereiral6 U-Net 
Havaeil6 U-Net 


Proposed 


Table 2. Performance variations in methods on BraTS 2015 dataset 
Dice Score Sensitivity Specificity 

WT CT ET WT CT ET WT CT ET 
HGG 0.81 0.70 0.67 0.80 0.73 0.74 0.82 0.73 0.63 
LGG 0.60 0.49 0.00 0.65 0.69 0.66 0.69 0.67 0.63 
HGG 0.81 0.73 0.67 0.57 0.63 0.66 0.75 0.72 0.74 
LGG 0.81 0.53 0.19 0.72 0.75 0.75 0.70 0.73 0.75 
HGG 0.82 0.73 0.68 0.83 0.82 0.86 0.85 0.77 0.79 
LGG 0.82 0.57 040 0.82 0.76 0.78 0.84 0.74 0.77 


Pereiral6 U-Net 
Havaeil6 U-Net 


Proposed 


Input Image Label U-Net Method Proposed Method 


Figure 4. Comparison results of sample images from left to right, ground truth, U-Net, and proposed method 
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5. CONCLUSION 

This new algorithm makes use of deep recurrent level sets and integrates the advantages of both deep 
learning for carrying out brain tumour segmentation as an automatic process. The existing standard models 
have also been briefly discussed to achieve contextual orientation. The results obtained confirm that by 
integrating level sets and recurrent FCN architectures, the proposed DRLs offers superior solution through its 
robustness against outliers, speed and consistent while segmenting core tumour. Additionally, DRLs improves 
the speed of segmenting brain tumours to a large extent, thus making it a practical solution. Consequently, the 
results demonstrate that the proposed methods show state-of-the-art performance in all three tasks with 
sufficient robustness to handle data from multiple datasets. In future, we plan extensions to the proposed 
architecture by integrating whole slide image and molecular genetic features for tumour classification 
following new WHO criterion4. 
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